1. Field of the Invention
The present invention relates to a technique of obtaining placement information of a 6 degrees of freedom (6-DOF) position and orientation sensor attached to an object with respect to the object.
2. Description of the Related Art
In recent years, studies on mixed reality for seamless fusion of a physical space and a virtual space have been actively conducted. An image display device used in a mixed reality system is mainly realized by a video see-through method. In the image display device using the video see-through method, an image of a virtual space (e.g., a virtual object or text information rendered by computer graphics) generated in accordance with the position and orientation of an image capturing device, such as a video camera, is superimposed on an image of a physical space captured by the image capturing device, and then the mixed image is displayed.
In the mixed reality system, it is important to perform registration between a physical space and a virtual space with high accuracy. Much effort has been made for the registration. A problem about registration in mixed reality leads to a problem about obtaining relative position and orientation between an object on which virtual information is to be superimposed and an image capturing device (hereinafter referred to as the position and orientation of the object with respect to the image capturing device).
As a method for solving the problem, the following methods have traditionally been proposed in a field of photogrammetry or the like (e.g., see R. M. Haralick, C. Lee, K. Ottenberg, and M. Nolle: Review and analysis of solutions of the three point perspective pose estimation problem, International Journal of Computer Vision, vol. 13, no. 3, pp. 331-356, 1994; and D. G. Lowe: Fitting parameterized three-dimensional models to images, IEEE Transactions on PAMI, vol. 13, no. 5, pp. 441-450, 1991).
A plurality of indices are placed or set on an object. The placement of the indices in the coordinate system of the object is known. Then, the position and orientation of the object with respect to an image capturing device are obtained by using the known information, that is, three-dimensional coordinates of the indices in the object coordinate system, and coordinates of projected images of the indices in an image captured by the image capturing device.
However, the registration method using only image information is easily affected by a change in lighting condition and does not have sufficient stability. For this reason, a 6-DOF position and orientation sensor capable of obtaining a stable measurement value without depending on lighting condition, such as a magnetic sensor, is attached to each of the object and the image capturing device. Under this state, the position and orientation of the object with respect to the image capturing device are calculated by using a position and orientation measurement value of the object and a position and orientation measurement value of the image capturing device.
Also, a hybrid position and orientation measuring method has been proposed (e.g., see Uchiyama, Yamamoto, and Tamura: A robust registration method for merging real and virtual worlds—Combining 6 DOF sensor and vision algorithm, Monographs of The Virtual Reality Society of Japan, vol. 8, no. 1, pp. 119-125, 2003). In this method, a measurement value of a 6-DOF position and orientation sensor, such as a magnetic sensor, is calibrated by using coordinates of projected images of indices in an image captured by an image capturing device.
6-DOF position and orientation sensors are attached to an object and an image capturing device, respectively, and relative position and orientation between the object and the image capturing device are obtained. In order to render a virtual object on the object without causing displacement by using the relative position and orientation, placement information of the 6-DOF position and orientation sensor with respect to the object to be measured, that is, relative position and orientation need to be accurately calculated. Conventionally, an operation of obtaining placement information of the 6-DOF position and orientation sensor with respect to the object—hereinafter referred to as calibration information, has been performed manually.